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1 Introduction 

Since the advent of magnetic resonance imaging (MRI) and computed tomogra- 
phy (CT) scanners around the early seventies, and the consequent ubiquitousness of 
medical volume data, medical visualization has undergone significant development 
and is now a primary branch of Visualization. It finds application in diagnosis, for 
example virtual colonoscopy, in treatment, for example surgical planning and guid- 
ance, and in medical research, for example visualization of diffusion tensor imaging 
data. Although the field of medical visualization only established itself with this 
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name in the late eighties |57], we shall see in the next section that already in the 
seventies there were published examples of computer-generated images, based on 
medical data and used for medical applications. 

During the past decades, medical image acquisition technology has undergone 
continuous and rapid development. It is now possible to acquire much more complex 
data than ever before. For example, in High Angular Resolution Diffusion Imaging 
(HARDI), forty or more diffusion-weighted volumes are acquired in order to calcu- 
late and visualize water diffusion and, indirectly, structural neural connections in the 
brain ]70) . In fMRI-based full brain connectivity, time-based correlation of neural 
activity is indirectly measured between all pairs of voxels in the brain, thus giving 
insight into the functional neural network J24). Moreover, the questions that users 
attempt to answer using medical visualization have also become significantly more 
complex. 

In this paper, we first give a high-level overview of medical visualization devel- 
opment over the past 30 years, focusing on key developments and the trends that 
they represent. During this discussion, we will refer to a number of key papers that 
we have also arranged on the medical visualization research timeline shown in fig- 
ure[T] Based on the overview and our observations of the field, we then identify and 
discuss the medical visualization research challenges that we foresee for the coming 
decade. 



2 Thirty-year Overview of Medical Visualization 

Already in 1978, Sunguroff and Greenberg published their work on the visualization 
of 3D surfaces from CT data for diagnosis, as well as a visual radio-therapy plan- 
ning system, also based on CT data (64). Five years later, Vannier et al. published 
their results developing a system for the computer-based pre-operative planning of 
craniofacial surgery |7T[ . By this time, they had already used and evaluated their 
surgical planning system in treating 200 patients. The system was based on the ex- 
traction and visualization of 3D hard and soft tissue surfaces from CT data. Through 
the integration of an industrial CAD application, it was also possible to perform de- 
tailed 3D measurements on the extracted surfaces. 



2.1 Practical and Multi-modal Volume Visualization 

In 1986, Hohne and Bernstein proposed using the gray-level gradient to perform 
shading of surfaces rendered from 3D CT data [26|. In 1987, Lorensen and Cline 
published the now famous Marching Cubes isosurface extraction algorithm, which 
enabled the fast and practical extraction of 3D isosurfaces from real-world medi- 
cal data. In the year thereafter, Levoy introduced the idea of volume raycasting in 



May [47], and Drebin et al. in August [19]. Although medical volume visualization 
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1978 - Sunguroff and Greenberg: CT and 3D surfaces for diagnosis and radiotherapy planning |64| . 
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1983 - Vannier et al.: 3D surfaces from CT for planning of craniofacial surgery 1 71 1. 

1986 - Hohne and Bernstein: Shading 3D Images from CT using gray-level gradients j26j . 

1987 - Lorensen and Cline: Marching Cubes (49) . 

1988 - Levoy publishes Direct Volume Rendering paper in May [47| , Drebin et al. in August [19| . 

Multi-modal volume rendering by Hohne et al. [27| . 



1993 - Altobelli et al: Predictive simulation in surgical planning Gerig et al.: Vessel visualization |23| . 

1994 - Basser et al.: Diffusion Tensor Imaging (5j. 

1995 - Hong et al: 3D Virtual Colonoscopy [28] . 

1996 - Behrens et al. et al. : Visualization of Dynamic Contrast-Enhanced MRI mammography data (time-varying) ^j. 
1998 - Basser et al: DTI Tractography |4|6| . 

2000 - Ebert and Rheingans - Medical Volume Illustration [20 . 

2001 - Tory et al : Multi-timepoint MRI |69) . 

2003 - Kriiger and Westermann: GPU raycasting [45| . 



2007 - Blaas et al: Multi-Field Medical Visualization (5). 

2008 - Wang et al: LifeLines2 - multi-subject electronic health records |73] , 

2010 - Steenwijk et al: Cohort studies - multi-subject imaging and metadata [63]. 

Fig. 1 Timeline with a subset of medical visualization papers showing the progression from scalar 
volume datasets through time-dependent data to multi-field and finally multi-subject datasets. This 
timeline is by no means complete, instead attempting to show a representative sample of papers 
that represent various trends in the development of the field. 

was possible before these publications, as witnessed by a number of publications, 
previous techniques were either not as fast or yielded less convincing results. With 
the introduction of Marching Cubes and volume raycasting, volume visualization 
became a core business of visualization and medical visualization for the years to 
come. 

Up to this point, research had focused on uni-modality data, primarily CT. How- 
ever, already in 1988 the first multi-modal volume rendering paper was published by 
Hohne et al, in which they demonstrated the registration and combined visualiza- 
tion of CT and MRI. A great deal of work has been done since then on the theory and 
applications of multi-modal volume visualization. The first notable example is the 
work of Cai and Sakas in 1999 where they classified voxel- voxel multi -modal vol- 



4 



Botha, Preim, Kaufman, Takahashi, Ynnerman 



ume rendering techniques according to the volume rendering pipeline stage where 
they take place [13 ]. The three classes are image level, where two volume renderings 
are combined pixel-by-pixel, accumulation level, where looked up samples along 
the ray are combined, and illumination model level, where the illumination model is 
adapted to process two volume samples directly. The second example we mention 
is a convincing application of multi-modal volume rendering for the planning of 
neurosurgical interventions, where MRI, CT, fMRI, PET and DSA data are all com- 
bined in an interactive but high quality visualization for the the planning of brain 
tumor resection 1 8 1 . 



2.2 Therapy Planning, Predictive Simulation, and Diagnosis 

Therapy planning was one of the first real applications of medical visualization and 
remains important to this day. In 1993, Altobelli et al. published their work on using 
CT data to visualize the possible outcome of complicated craniofacial surgery QJ. 
By manually repositioning soft tissue fragments based on the bony surfaces under 
them, in certain cases taking into account bone-skin motion ratios from literature, 
the expected outcome of a craniofacial procedure could be visualized. Although 
still rudimentary, this could be considered one of the earliest cases of predictive or 
outcome simulation integrated with visualization for surgical planning. The idea of 
predictive simulation, or predictive medicine, was further explored by Taylor et al. 
for cardiovascular surgery (66). 

With the introduction of virtual colonoscopy (VC) in 1995 [28], medical visual- 
ization also gained diagnosis as an important medical application, namely screening 
for colon cancer. VC combines CT scanning and volume visualization technologies. 
The patient abdomen is imaged in a few seconds by a multi-slice CT scanner. A 
3D model of the colon is then reconstructed from the scan by automatically seg- 
menting the colon and employing "electronic cleansing" of the colon for computer- 
based removal of the residual material. The physician then interactively navigates 
through the volume rendered virtual colon employing camera control mechanisms, 
customized tools for 3D measurements, "virtual biopsy" to interrogate suspicious 
regions, and "painting" to support 100% inspection of the colon surface |29) . VC is 
rapidly gaining popularity and is poised to become the procedure of choice in lieu 
of the conventional optical colonoscopy for mass screening for colon polyps - the 
precursor of colorectal cancer. Unlike optical colonoscopy, VC is patient friendly, 
fast, non-invasive, more accurate, and cost-effective procedure for mass screening 
for colorectal cancer. 

VC technologies gave rise to the computer-aided detection (CAD) of polyps, 
where polyps are detected automatically by integrating volume rendering, confor- 
mal colon flattening, clustering, and "virtual biopsy" analysis. Along with the re- 
viewing physician, CAD provides a second pair of "eyes" for locating polyps | [30| . 
This work was also the basis for many other virtual endoscopy systems, such as 
virtual bronchoscopy, virtual cystoscopy, and virtual angioscopy. A careful integra- 
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tion of image analysis (e.g., segmentation, skeletonization), with efficient rendering 
(e.g., occlusion culling) and interaction (e.g., camera control based on predefined 
paths) are major ingredients of such systems |3]|29). 



2.3 Multi-field Data 

Diffusion Tensor Imaging, or DTI, is an MRI-based acquisition modality, intro- 
duced in 1994 by Basser et ai, that yields 3x3 symmetric diffusion tensors as its 
native measurement quantity [5|. The tensors represent the local diffusion of wa- 
ter molecules, and hence indirectly indicate the presence and orientation of fibrous 
structures, such as neural fiber bundles or muscle fibers. Already in this first paper, 
the authors employed 3D glyphs to visualize the eigensystems represented by the 
tensors, as shown in figure [2] 

Basser and his colleagues were also some of the first to extract and visualize fiber- 
tract trajectories from DTI data of the brain (4j|6j, thus linking together the point 
diffusion measurements to get an impression of the global connective structures in 
the brain. With DTI it was encouraging to see that the first visualization efforts were 
initiated by the scientists developing this new scanning modality themselves. Early 
work by the visualization community includes tensor lines for tractography |76) and 
direct volume rendering of DTI data p8][39) . 



IDL2 




Fig. 2 Effective diffusion ellipsoid glyphs from a 2D region of interest of a DTI dataset of a cat 
brain. Image taken from 
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Importantly, DTI serves as one of the first examples of natively multi-field medi- 
cal data, that is medical data with multiple parameters defined over the same spatio- 
temporal domain. The advent of DTI initiated a whole body of medical visualization 
research dedicated to the question of how best to visually represent and interact with 
diffusion tensor data in particular and multi-field medical data in general. The 2007 
paper by Blaas et al. presented a visual analysis-inspired solution to this problem 
based on linked physical and feature space views [9|. 



2.4 Time-varying Data 

Time-varying medical volume data visualization made its entrance in 1996 with 
work by Behrens et al. on supporting the examination of Dynamic Contrast- 
Enhanced MRI mammography data with the display of parameter maps, the se- 
lection of regions of interest (ROIs), the calculation of time-intensity curves (TICs), 
and the quantitative analysis of these curves [7|. In 2001, Tory et al. presented meth- 
ods for visualizing multi-timepoint (1 month interval) MRI data of a multiple scle- 
rosis (MS) patient, where the goal was to study the evolution of brain white matter 
lesions over time |69) . Methods used included glyphs, multiple isosurfaces, direct 
volume rendering and animation. Coto et al. applied multiple coupled views, in- 
cluding linked cursors and brushing on 3D renderings and scatterplots, to dynamic 
contrast-enhanced MRI (DCE-MRI) mammography data p6| . 



2.5 Illustrative Visualization 

Illustrative visualization is primarily motivated by the attempt to create renditions 
that consider perceptual capabilities of humans. As an example, humans infer in- 
formation about shapes not only from realistic shading but also from appropriate 
hatching and from outlines that support the mental separation of nearby objects ren- 
dered in similar colours. 

Illustrative visualization is related to the term Non-Photorealistic Rendering in 
computer graphics, or NPR for short. The term NPR was used since around 1990 
when the seminal paper of Saito et al. clearly illustrated that complex 3D shapes 
could be rendered more comprehensible by using certain feat ure lines |6T|. Com- 
pared to NPR, illustrative visualization is the more focused term that covers render- 
ing techniques serving clear visualization goals, namely to convey shape informa- 
tion efficiently. In medical visualization, either surfaces or volume data are rendered 
in illustrative styles. For illustrative volume rendering, the term volume illustration 
was introduced by Ebert et al. in 2000 |20|. Boundary enhancement based on gra- 
dient approximation 1 17 1 and curvature-based transfer functions |40] are landmarks 
in illustrative medical visualization. Tietjen et al. applied silhouettes and other fea- 
ture lines for various scenarios in liver surgery planning |68|. Besides silhouettes, 
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stippling and probably even more hatching has great potential to reveal details of 
shapes (32). 

Later, Bruckner et al. made a number of important contributions that support 
depth and shape perception with adapted transfer functions. In particular, they con- 
sidered the peculiarities of interactive exploration of 3D datasets and elaborated on 
the idea of preserving essential context information pOj - fLT j. These and later refine- 
ments are integrated in the VolumeShop-system that is publicly available and used 
by several research groups 



2.6 Multi-subject Data 

Medical visualization has also started to work on the problem of dealing with multi- 
subject data. These are datasets that include measurements, including imaging, of 
more than one subject. The goal is to be able to extract patterns that affect sub- 
groups of the whole collection, for example to explore which aspects of the data 
correlate with a specific disease outcome. Examples of this type of work include 
LifeLines2, an information visualization approach to visualize and compare mul- 
tiple patient histories or electronic medical records (73]. More recently, work has 
been done on the interactive visualization of the multi-subject and mixed modality 
datasets acquired by medical cohort studies |63|. In these studies, mixed modality 
data, including imaging, genetics, blood measurements and so on, is acquired from 
a group of subjects in order to understand, diagnose or predict the clinical outcome 
of that group. Steenwijk et al. demonstrated that it was possible to create a highly 
interactive coupled view visualization interface, integrating both information and 
scientific visualization techniques, with which patterns, and also hypotheses, could 
be extracted from the whole data collection. 



3 Challenges in Medical Visualization 

3.1 Advances in Data Acquisition 

Toshiba's 320-slice CT scanner, the Aquilion One, was introduced in 2007. It is 
able to acquire five 320 slice volumes per second [31 1 and can thus image a beating 
heart. Rapid and continuous advances in the dynamic nature and sheer magnitude 
of data in this and other mainstream medical imaging necessitates improvements to 
existing techniques in terms of computational and perceptual scalability. 

High Angular Resolution Diffusion Imaging (HARDI) [70] and Diffusion Spec- 
trum Imaging (DSI) [25 1 datasets contain hundreds of diffusion-weighted volumes 
describing the diffusion of water molecules and hence indirectly the orientation of 
directed structures such as neural fiber bundles or muscle fibers. This is a rather 



8 



Botha, Preim, Kaufman, Takahashi, Ynnerman 



extreme example of multi-field medical data that is becoming more relevant in both 
medical research and clinical application. Completely new visual metaphors are re- 
quired to cope with the highly multi-variate and three-dimensional data of diffusion 
weighted imaging in particular and many other new imaging modalities in general. 

Molecular imaging enables the in vivo imaging of biochemical processes at 
macroscopic level, meaning that, for example, pathological processes can be stud- 
ied and followed over time in the same subject long before large-scale anatomical 
changes occur. Examples are bioluminescence (BLI) and fluorescence (FLI) imag- 
ing, two molecular imaging modalities that enable the in vivo imaging of gene ex- 
pression. Molecular imaging yields datasets that vary greatly in scale, sensitivity, 
spatial-temporal embedding and in the phenomena that can be imaged. Each per- 
mutation brings with it new domain-specific questions and visualization challenges. 
Up to now, most of the visualization research has been focused on small animal 
imaging ]4Tj|42) , but due to its great diagnostic potential, molecular imaging will 
see increased application in humans. 

The integration of microscopy imaging is an essential task for the future, where 
data handling, interaction facilities but also more classical rendering tasks such as 
transfer function design become essential. With more and more large scale and 
3D microscopy data available, there are many opportunities for visualization re- 
searchers. Recent examples include techniques for interactively visualizing large- 
scale biomedical image stacks demonstrated on datasets of up to 160 gigapixels |34| 
and tools for the interactive segmentation and visualization of large-scale 3D neuro- 
science datasets, demonstrated on a 43 gigabyte electron microscopy volume dataset 
of the hippocampus (33) . 

With these examples, we hope to have illustrated that advances in image acqui- 
sition are continuous, and due to the increasing demands of modern society are 
accelerating. Each new advance in imaging brings potentially greater magnitudes 
and exotic new types of data leading to new challenges for medical visualization. 



3.2 Heterogeneous Display and Computing Devices 

Mobile devices, in particular the Apple products iPad and iPhone, are extremely 
popular among medical doctors and indeed solve some serious problems of desk- 
top devices in routine clinical use. In particular, bedside use of patient data is an 
essential use case for medical doctors of various disciplines. 

Meanwhile several mobile devices are equipped with powerful graphics cards 
and using the OpenGL ES (Embedded Systems) standard, they are able to provide 
high-quality interactive rendering. Although the performance still tails that of mod- 
ern desktop devices, slicing medical volume data and 3D rendering is feasible |52|. 

The rapid and widespread use of mobile devices also made gesture input popu- 
lar. In particular, multi-touch interaction is considered an intuitive interaction since 
many potential users know a variety of gestures from their everyday activities with 
smart phones. Therefore multitouch interaction is also incorporated in large displays 
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in medical use, e.g. the Digital Lightbo^by BrainLab and the multi-touch table of 
Lundstrom et al. J50l. 



3.3 Interactive Image Segmentation 

Image segmentation is important in clinical practice, for example, diagnosis and 
therapy planning, and also in image-based medical research. In these applications, 
segmentation is complicated by the great deal of variation in image acquisition, 
pathology and anatomy. Furthermore, in matters of diagnosis or therapy planning, 
the accuracy of the segmentation can be critical. It comes as no surprise that user 
interaction is often required to help ensure the quality of the results, by initializing 
an image processing method, checking the accuracy of the result or to correct a 
segmentation J55) . 

A well-known interactive segmentation technique is the live-wire or intelligent 
scissors pi) . These ideas were later extended and applied to medical images pT[ . 
More recently, visualization has been applied to the challenge of explicitly dealing 
with the uncertainty inherent in interactive 3D segmentation p6][60) . 

Medical visualization research often combines elements of image analysis, graph- 
ics and interaction, and is thus ideally equipped to address the challenge of devel- 
oping and validating effective interactive segmentation approaches for widespread 
use in medical research and practice. 



3.4 Topological Methods 

Topological data representation has played an important role in medical visualiza- 
tion, since it can allow us to segment specific features such as human organs and 
bones from the input medical datasets systematically and identify their spatial rela- 
tionships consistently. Actually, such topological concepts have been already intro- 
duced in the earlier stage of medical visualization. For example, the 3D surface of a 
human cochlea was reconstructed from a series of 2D CT cross-sectional images by 
identifying correct correspondence between the cross-sectional contours |62[ . 

Topological approaches have also been extended to analyze 3D medical volume 
data. Contour trees fZ) have been employed for designing transfer functions in order 
to illuminate human organs and bones systematically since the associated anatomi- 
cal structure can be effectively captured as topological skeletons of isosurfaces j75). 
Spatial relationships between bones and the position of aneurysm were successfully 
extracted respectively from CT and angiographic datasets using a variant of contour 
tree fl5) . Interesting features in medical volume data can be visually analyzed us- 
ing an optimal camera path, which can be obtained by referring to the topological 



http : //www . brainlab . com/art /284 1/ 4/surgical-pacs-access/ 
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structure of human organs (65) (see figure [3}. Topological methods are now being 
developed for visualizing multi-variate and high-dimensional datasets, and thus po- 
tentially for analyzing tensor fields obtained through DT-MRI, multi-subject data in 
group fMRI studies, and time-varying data measured by high-speed CTs. 




3.5 Integration of Simulation Models 

In their 1999 predictive medicine paper, Taylor et al. argued that surgical planning 
should not only address questions of surgical approach but also of the expected 
outcome, for example predicted future states such as the efficacy of a treatment 
option or the performance of an implant [66|. Medical visualization approaches be- 
come significantly more valuable when enhanced with simulation models that help 
to predict the outcome of a disease process or therapeutic procedure, or that en- 
rich measured data with expected physiological phenomena. Examples besides the 
blood flow simulations of Taylor et al. include interactive skeletal range of mo- 
tion (43) and biomechanical stress [18] simulation models for implant planning in 
orthopedics and nasal airflow simulation for reconstructive rhinosurgery (77) . 

The integration of these predictive models, although potentially valuable, brings 
with it new challenges. The addition of potentially complex and dynamic simula- 
tion output data to existing visualizations requires new visual representation tech- 
niques. Furthermore, for the simulation results to be maximally useful, the models 
should be tightly coupled to and steered by the user's interaction with the medical 
visualization. Finally, most simulations yield data with a certain degree of inherent 
uncertainty. The role of this uncertainty should be fully explored and it should be 
carefully but explicitly represented as an integral part of the visualization. 
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3.6 Mappings and Reformations 

In 2002, the American Heart Association proposed a standardised segmentation and 
accompanying 2D bull's eye plot (see figure [4| of the myocardium, or heart mus- 
cle, of the left heart ventricle [14]. This 2D plot is a simple but great example of 
reducing complex 3D data to a standardized 2D representation that greatly facili- 
tates the interpretation of that data. Another well-known example is that of curved 
planar reformation, or CPR, where volume data is sampled along a curved plane 
following the trajectory of a blood vessel or other tubular structure, thus enabling 
the study of the vessel and its surroundings with the minimum of interaction p6| . 
Other good examples of reformation can also be found in brain flattening p2) and 
colon unfolding | [72) . 




Fig. 4 On the left, the American Heart Association standardized 2D bull's eye plot (BEP) of the 
left ventricle of the heart |14| . On the right, the volumetric bull's eye plot, a modernized version 
by Termeer et al. |67) . 

Recently, the idea of intelligently reformatting or mapping 3D data was further 
explored by Neugebauer et al. with aneurysm maps for the visualization of complex 
blood flow simulation data on the inside surfaces of aneurysms J53) and by Rieder et 
al. with their tumor maps for the post-operative assessment of radiofrequency abla- 
tion therapy J59) . These types of reformations and mappings entail that more effort 
has to be put into carefully designing simplified, usually 2D, representations of com- 
plex 3D data, as opposed to for example the relatively straight-forward projection of 
volume data. The resultant visualization, if done right, requires less or no interaction 
and by definition avoids a number of problems inherent in 3D representations |74|. 
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3.7 Illustrative Visualization in Medicine 



For a long time, it was not possible to apply illustrative visualization techniques in 
practice due to performance constraints. With advances in graphics hardware and 
algorithms, such as GPU raycasting |45j, it is now feasible from a computational 
standpoint. Now that computational problems have been largely solved, illustrative 
visualization approaches have to be finetuned and evaluated for diagnostic and treat- 
ment planning purposes. Recent examples of such work include the simulation of 
crepuscular rays for tumor accessibility planning [37] and multi-modal illustrative 
volume rendering for neurosurgical tumor treatment (58). 

Illustrative medical visualization becomes increasingly important when visual- 
izations become more complex and multi-modal, integrating functional (measured 
and simulated), anatomical information and for example surgical instruments. Illus- 
tration techniques enable visual representations to be simplified intelligently by the 
visualization designer, whilst still communicating as much information as possible. 
An example of this is the work of Zachow et al. on the visualization of nasal air 
flow simulation where boundary enhancement was used as an illustrative technique 
to convey the simulated flow and the anatomy simultaneously J77) . 



3.8 Hyper-realism 

Analogous to the case of illustrative visualization, the rapid development in graph- 
ics hardware and algorithms has now enabled the interactive rendering of medical 
imaging datasets with physically-based lighting (44). Figure [5] shows an example of 
such a visualization. These techniques make possible the simulation of an arbitrary 
number of arbitrarily shaped and textured lights, real shadows, a realistic camera 
model with lens and aperture, and so forth, all at interactive rates. 

These techniques enable not only photo-realism, but also a technical form of 
hyper-realism in art, where it is possible to enhance visualizations with additional 
realistic detail in order to better convey information. Whilst there are strong indi- 
cations that for example global illumination and shadows can have a positive effect 
on task performance in normal volume rendering [48 ], the possibilities and value of 
hyper-realistic effects in medical visualization need to be explored. 



3.9 Visual Analysis in Healthcare 

Visual analysis is becoming an essential component of medical visualization due 
to the rapidly growing role and availability of complex multi-dimensional, time- 
varying, mixed-modality, simulation and multi-subject datasets. In our view, the 
magnitude and especially the heterogeneity of the data necessitate the use of visual 
analysis techniques. 
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Fig. 5 Two examples of interactive visualizations made with the volume Tenderer of Kroes et 
al. (44 1 . Through the use of GPUs, physically-based lighting has become possible in an interactive 
volume rendering setting, enabling increased realism through soft shadows, depth of field and in 
this case mixed phase function and BRDF surface scattering. 



Existing examples involving time-varying data include the work of Coto et al. 
on DCE-MRI mammography |16| and Oeltze et al. on perfusion data in general 
and MR perfusion of the brain in particular |54|. Blaas et al. applied visual analy- 
sis techniques to multi-modal medical data, whilst Zachow et al. focused on nasal 
airflow simulation data combined with anatomical information |77) - 

There is great potential for visual analysis in medical visualization, with clinical 
applications including advanced diagnosis and medical research and, even more im- 
portantly, treatment planning and evaluation, e.g. radio therapy planning and post- 
chemotherapy evaluation. The new Visual Analysis in Healthcare (VAHC) work- 
shops that were held at IEEE VisWeek in 2010 and 2011 underline the emerging 
importance of this research direction. 



3.10 Population Imaging 

In population imaging, medical image data and other measurements are acquired 
of a large group of subjects, typically more than one thousand, over a longer pe- 
riod, typically years, in order to study the onset and progression of disease, general 
aging effects, and so forth in larger groups of people. Examples include the Rotter- 
dam Scan Study focusing on neuro-degeneration [46 1 and the Study of Health In 
Pomerania (SHIP) focusing on general health ]35) . 
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This application domain is an extreme example of multi-subject medical visual- 
ization discussed in section [2j integrating large quantities of heterogeneous, multi- 
modal and multi-timepoint data acquired of a large group of subjects. The scientists 
running these studies usually do not formulate strictly-defined hypotheses before- 
hand, instead opting for meticulous data acquisition, followed by an extended pe- 
riod of analysis in order to extract patterns and hypotheses from the data. Recently, 
Steenwijk et al. set the first steps for the visualization of population imaging by 
applying visual analysis techniques to cohort study imaging data |63) . The extreme 
heterogeneity and magnitude of the data, coupled with the explorative nature of the 
research, renders this a promising long-term application domain for visual analysis 
and medical visualization. 



4 Conclusions 

In this chapter, we gave a compact overview of the history of medical visualization 
research, spanning the past 30 years. Based on this history and on our own observa- 
tions working in the field, we then identified and discussed the research challenges 
of the coming decade. 

Our discussion of classic medical visualization problems related to efficient and 
high quality display of one static dataset was brief. We devoted more space to 
data that change over time, to the integration of anatomy with simulation and fi- 
nally to cohort studies. We refer to problems where such time-dependent and high- 
dimensional data are employed as MedVis 2.0 problems. While the classic problems 
are - from an application perspective - solved, there are many research opportuni- 
ties in MedVis 2.0 problems. These data are significantly more difficult to analyze, 
to process and to visualize. Time-dependent MRI data, e.g., exhibit all artifacts of 
static MRI data but a number of additional artifacts, e.g. due to motion. Integrated 
analysis and visualization is a key feature of MedVis 2.0 solutions. In general, suc- 
cessful solutions to these problems require a considerably deeper understanding of 
the medical background and thus favor close collaborations with medical doctors 
over merely having access to medical data. 
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